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ABSTRACT 

When  flying  an  airplane,  landing  is  arguably  the  most  difficult  task  a  pilot  can  do.  This  applies  to  pilots  of  all  skill  levels 
particularly  as  the  level  of  complexity  in  both  the  aircraft  and  environment  increase.  Current  navigational  aids,  such  as 
an  instrument  landing  system  (ILS),  do  a  good  job  of  providing  safe  guidance  for  an  approach  to  an  airfield.  These  aids 
provide  data  to  primary  flight  reference  (PFR)  displays  on-board  the  aircraft  depicting  through  symbology  what  the 
pilot’s  eyes  should  be  seeing.  Piloting  an  approach  under  visual  meteorological  conditions  (VMC)  is  relatively  easy 
compared  to  the  various  complex  instrument  approaches  under  instrument  meteorological  conditions  (IMC)  which  may 
include  flying  in  zero-zero  weather.  Perhaps  the  most  critical  point  in  the  approach  is  the  transition  to  landing  where  the 
rate  of  closure  between  the  wheels  and  the  runway  is  critical  to  a  smooth,  accurate  landing.  Very  few  PFR’s  provide  this 
flare  cue  information.  In  this  study  we  will  evaluate  examples  of  flare  cueing  symbology  for  use  in  landing  an  aircraft  in 
the  most  difficult  conditions.  This  research  is  a  part  of  a  larger  demonstration  effort  using  sensor  technology  to  land  in 
zero-zero  weather  at  airfields  that  offer  no  or  unreliable  approach  guidance.  Several  problems  exist  when  landing 
without  visual  reference  to  the  outside  world.  One  is  landing  with  a  force  greater  than  desired  at  touchdown  and  another 
is  landing  on  a  point  of  the  runway  other  than  desired.  We  compare  different  flare  cueing  systems  to  one  another  and 
against  a  baseline  for  completing  this  complex  approach  task. 

Keywords:  Enhanced  vision,  flight  symbology,  flight  guidance,  situation  awareness,  military  transport 

1.  INTRODUCTION 


1.]  Background 

This  paper  presents  the  results  of  the  first  of  three  studies  in  support  of  the  Autonomous  Approach  and  Landing 
Capability  (AALC)  demonstration  program.  The  AALC  program  was  initiated  to  advance  the  development  of  an 
enhanced  vision  system  for  safely  conducting  self  contained  approach  and  landing  operations  in  poor  visibility 
conditions.  The  issues  addressed  by  this  and  the  other  studies  were  cited  as  concerns  in  a  1998  flight  test  program The 
two  issues  addressed  here  are:  1)  excessive  sink  rate  attributed  to  low  quality  sensor  imagery,  and  2)  touchdowns  outside 
of  the  desired  touchdown  zone.  Each  of  these  concerns  was  raised  previously  by  pilots  because  of  the  relatively  low 
quality  images  provided  by  the  sensor  and  the  symbology  at  hand. 

1.2  Test  Item  Description 

The  Transport  Aircraft  Cockpit  (TRAC)  is  a  reconfigurable,  three  seat  (pilot,  copilot,  flight  engineer)  transport  aircraft 
cockpit  research  simulator.  Wide-angle  collimating  windows  are  used  for  displaying  out-the-window  scenes.  Head  down 
instrument  formats  are  displayed  using  three  21"  monitors  across  the  front  of  the  cockpit.  Active  matrix  liquid  crystal 
displays  are  also  available  as  head  down  display  devices.  Several  aero-  and  feel  models  (e.g.  C-141,  C-17)  are  available 
to  provide  aircraft  flying  characteristics.  The  simulation  also  has  the  flexibility  to  switch  between  a  center  yoke,  a  center 
C- 17-style  stick,  or  a  side  stick  controller  for  flight  control  inputs.  For  this  evaluation,  TRAC  was  configured  with  the 
three  display  configurations,  a  C-17  aeromodel,  and  the  center  C- 17-style  flight  control  stick. 

Cockpit  display  formats  and  out-the-window  scene  graphics  were  generated  by  Linux-based  PCs.  Visual  scene 
generation  occurred  at  rates  varying  from  25Hz  to  30Hz  depending  upon  scene  complexity.  Graphics  programming  was 
accomplished  in  C/C++  and  Fortran  languages,  with  C++  being  the  program  of  choice  for  visual  scene  rendering. 

The  AALC  system  simulated  in  TRAC  consisted  of  a  digital  light  engine  (DLE)  head-up  display  (HUD)  and  94  GHz 
active  millimeter  wave  (MMW)  radar.  The  HUD  symbology  displayed  consisted  of  the  primary  flight  reference  as 
described  in  the  C-17  “dash  1”^.  Because  TRAC  does  not  have  a  HUD,  the  symbology  was  displayed  on  the  out-the- 


window  scene.  An  effects-based  simulation  of  MMW  radar  imagery  was  written  in-house  with  assistance  from 
AFRL/SN  and  the  prime  contractor. 

1.3  Test  Objectives 

a.  Evaluate  the  ability  of  the  candidate  displays  to  provide  the  pilot  with  sufficient  cues  to  touchdown  within  the 
desired  landing  zone. 

b.  Evaluate  the  ability  of  the  candidate  displays  to  provide  the  pilot  with  sufficient  cues  to  touchdown  with  an 
acceptable  sink  rate. 

c.  Evaluate  the  perceived  and  actual  situation  awareness  provided  by  the  candidate  displays  with  regard  to 
touchdown  location. 

d.  Evaluate  the  perceived  workload  experienced  by  pilots  when  using  the  candidate  displays. 

Weather  conditions  simulated  were  Visual  Meteorological  Conditions  (VMC)  -  no  restriction  to  visibility  -  and 
Instrument  Meteorological  Conditions  (IMC)  -  0'  ceiling  and  0'  runway  visual  range. 

1.4  Limitations 

TRAC  is  a  fixed-base  simulator;  therefore  vestibular  effects  of  conducting  these  operations  were  not  controlled.  Effects 
specific  to  obscuration  types  (e.g.  rain,  snow,  and  fog)  were  not  simulated. 

2.  TEST  AND  EVALUATION 


2.1.  Participants 

Twelve  pilots  of  varying  experience/qualification  levels  from  mobility  organizations  participated.  They  came  from  one 
of  three  USAF  C-17  wings,  the  C-17  SPO,  AFFTC,  and  the  Air  Force  Reserve. 

2.2.  Duration  of  Study 

Each  pilot  spent  one  day  (approximately  6  hours)  completing  simulator  “ground  school”  training,  simulator  flying 
training,  simulator  data  collection,  and  experiment  debriefing. 

2.3.  Design 

The  experimental  design  for  the  study  was  a  repeated  measures  three-factor  multivariate  analysis  of  variance.  Each 
participant  received  each  treatment  condition,  and  the  order  of  presentation  for  the  conditions  was  randomized  to  counter 
order  effects. 

2.4.  Independent  variables 

Three  factors  were  controlled,  two  with  three  levels  and  one  with  two  levels.  The  first  factor  was  HUD  configuration  and 
its  three  levels  were  minimum  impact  symbology  (MIS),  commercially  available  symbology  (CAS),  and  full 
outline/rising  runway  symbology  (FORRS).  The  second  independent  factor  was  runway  configuration  and  its  three 
levels  were  75',  150'  and  300'  widths;  all  runways  were  3000'  long.  The  final  factor  was  visibility  where  the  two  levels 
were  VMC  (unrestricted  ceiling  and  visibility)  and  IMC  (O'  ceiling  and  0'  visibility). 

2.5.  Dependent  variables 

Several  measures  were  collected  during  and  after  the  trials.  Landing  performance  data  was  collected  to  determine 
landing  technical  error.  Subjective  situation  awareness  data  were  collected  in  post-mission  and  post  experiment 
questionnaires.  Finally,  a  subjective  questionnaire  was  provided  to  solicit  specific  design  guidance  and  feedback. 

2.6.  Procedure 

Pilots  flew  36  simulated  landings  (roughly  5  minutes  each)  under  the  various  combinations  of  the  independent  variables 
HUD  X  Runway  width  x  Visibility  (3  x  3  x  2  x  2  replications).  The  scenario  began  approximately  4  miles  from  the 
runway  along  the  extended  centerline;  flightpath  error  data  collection  began  at  the  2  mile  (600'  AGL)  point. 


2.7.  Display  Configurations 

As  mentioned  there  were  three  configurations  that  combined  display  symbology  and  imagery.  The  three  configurations 
are  described  below.  Symbology  was  selected  for  its  purported  design  characteristics.  Runway  outlines  purport  to  aid 
aircrew  in  locating,  lining  up  with,  and  touching  down  in  the  proper  location  on  the  runway.  Flare  cues  are  designed  to 
give  the  pilot  better  information  on  when  to  begin  making  the  control  inputs  for  the  transition  to  landing^. 


2.7.1.  Configuration  1:  Minimum  Impact  Symbology  (MIS) 

The  first  was  treated  as  a  “minimum  impact”  baseline  configuration.  This  configuration  consisted  of  the  primary  flight 
information  from  the  C-17  HUD  overlaid  on  the  MMW  RADAR  image.  The  only  additional  symbology  in  configuration 
1  was  a  drift  angle  indicator  that  is  normally  displayed  on  the  head-down  display  (HDD)  in  the  C-17.  In  this  study  it  was 
displayed  on  the  HUD  (See  figure  1). 
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Figure  1.  Configuration  1  in  IMC 

The  task  was  to  use  the  dual  cue  flight  director  and  the  flight  path  marker  in  conjunction  with  the  underlying  imagery  to 
make  the  approach  and  landing.  Touchdown  location  cues  were  provided  via  the  imagery  only.  Vertical  velocity  cues 
were  provided  via  imagery  and  the  VVl  in  the  HUD.  In  VMC  the  underlying  imagery  was  an  unobstructed  daylight  view 
of  the  landing  zone  and  surround.  The  VMC  condition  also  included  the  landing  zone  marking  panels  according  to 
current  practice.  In  IMC  the  underlying  image  was  the  MMWR  image  as  shown  above.  Based  on  the  desire  of  the  user, 
no  analog  to  the  marking  panels  was  implemented  for  IMC. 

2.7.2.  Configuration  2:  Commercially  Available  Symbology  (CAS) 

The  second  display  configuration  used  the  same  imagery  as  configuration  1,  but  made  one  modification  to  the  C-17 
ITUD  as  well  as  two  additions.  The  flight  director  was  modified  from  the  dual-cue  pitch  and  bank  steering  bars  to  a 
single-cue  ball.  The  inputs  to  the  ball  were  equivalent  to  those  provided  to  the  steering  bars.  The  first  addition  was  a  flare 
cue  based  on  those  currently  available  on  commercial  aircraft  outfitted  with  HUDs.  This  cue  consists  of  a  circle  (the  FD) 
and  a  ‘plus’  sign  that  appears  at  105'  AGL.  As  the  aircraft  descends  the  glideslope,  the  task  of  the  pilot  is  to  track  the 
plus  sign  as  it  climbs  higher  in  the  display.  When  the  plus  sign  centers  in  the  circle  (at  55'  AGL)  the  pilot  should  initiate 
the  flare  maneuver. 

The  second  addition  was  a  symbolic  representation  of  the  runway  side  edges.  The  edge  lines  are  designed  to  function  as 
cues  for  lining  up  on  the  runway  and  to  reinforce  the  visual  cues  provided  by  the  image  (e.g.  perspective,  splay  angle, 
optical  flow).  See  figure  2. 
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Figure  2.  Configuration  2  in  IMC. 


2.7.3.  Configuration  3:  Full  Outline/Rising  Runway  Symbology  (FORRS) 

Expanding  on  the  idea  of  the  runway  edge  lines  from  the  previous  configuration,  a  full  runway  outline  was  provided 
(threshold,  departure  and  edges),  plus  a  runway  centerline  and  a  landing  zone  marker.  Just  as  the  runway  edge  lines  were 
expected  to  reinforce  the  underlying  imagery,  so,  too,  were  the  additional  runway  features  expected  to  reinforce  the 
imagery  and  the  edge  lines.  See  figure  3. 
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Figure  3.  Configuration  3  in  IMC  with  Flight  Director  (RA  >  105"). 

Configuration  three  reverted  to  the  dual-cue  flight  director  of  the  C-17  except  that  at  105'  AGL  a  mode  change  was 
effected  converting  the  pitch  steering  bar  into  a  rising  runway  cue.  Referenced  to  the  aircraft  waterline  symbol  the  rising 
runway  (nee  pitch  steering  bar)  centered  under  the  waterline  and  rose  at  a  rate  proportional  to  the  sink  rate.  The  bottom 


portions  of  the  waterline  symbol  represented  the  main  gear  so  that  when  the  rising  runway  contacted  the  bottom  'points’ 
of  the  waterline  symbol  the  main  gear  were  simultaneously  contacting  the  ground.  In  this  configuration  the  Flight  Path 
Marker  (FPM)  was  reduced  to  one  quarter  size  at  105'.  This  was  done  to  reduce  the  clutter  in  the  center  of  the  display. 
Configuration  three  again  used  the  same  underlying  imagery  as  the  other  two  configurations.  See  figure  4. 
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Figure  4.  Configuration  3  in  IMC  with  Rising  Runway  (RA<105')- 

2.8.  Measures 

To  determine  the  effect  of  the  different  display  configurations  several  parameters  were  measured. 

2.8.1.  Touchdown  location 

The  nominal  touchdown  location  was  established  as  the  center  of  the  landing  zone.  Deviations,  both  laterally  and 
longitudinally,  were  measured  from  the  center  of  the  touchdown  box.  These  data  were  also  used  to  score  the  landings  as 
pass/fail;  ‘pass’  if  they  were  in  the  box  and  ‘fail’  if  out.  Additionally,  lateral  and  longitudinal  errors  (in  feet)  from  the 
center  of  the  touchdown  zone  were  measured. 

2.8.2.  Vertical  Velocity 

A  continuous  measure  of  the  vertical  velocity  indicator  was  collected,  but  the  value  at  touchdown  was  used  for  these 
analyses. 

2.8.3.  Situation  Awareness 

Three  methods  were  used  to  collect  SA  data.  The  first  was  the  China  Lake  SA'*.  After  each  trial  the  pilot  would  refer  to 
the  China  Lake  SA  card  and  provide  a  rating  for  the  just  completed  run.  Available  ratings  ranged  from  1  (very  good)  to  5 
(very  poor).  Each  rating  was  accompanied  by  a  set  of  content  descriptors  to  help  verbally  anchor  the  ratings  (see  figure 
5). 


SA  SCALE  VALUE 

CONTENT 

VERY  GOOD 

1 

•  Full  luiowledge  of  aircraft  energy  stateftacticai  environment/mission 

•  Full  ability  to  anticipate/accomnnodate  trends 

GOOD 

2 

•  Full  Knowledge  of  aircraft  energy  state/lactical  environment/mission 

•  Partial  ability  to  anticipate /accommodate  trends 

•  No  task  shedding 

ADEQUATE 

3 

•  Full  Knov^edge  of  aircraft  energy  stateAactIcsl  environ menl/missjon 

•  Saturated  ability  to  anticipate/accommodate  trends 

•  Some  shedding  of  minor  tasks 

POOR 
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•  Fair  knowledge  of  aircraft  energy  stateAactlcat  environ ment/mtssion 

•  Saturated  ability  to  ant  id  pate/accommodate  trends 

•  Shedding  of  all  minor  tasks  as  well  as  many  not  essential  to  flight 
safely/mission  effectiveness  tasks 

VERY  POOR 

5 

•  Minimal  knowledge  of  aircraft  energy  stateAactlcat  environ meni/misslon 

•  Oversaturated  ability  to  aniicipate/accommodale  trends 

•  Shedding  of  ail  tasks  not  absolutely  essential  to  flight  safety/m tss ion 
effectiveness 

Figure  5.  China  Lake  Situation  Awareness  rating  card 

The  second  measure  was  also  collected  at  the  end  of  each  trial.  Here,  the  pilot  was  asked  to  place  a  pencil  mark  on  graph 
paper  where  the  touchdown  zone  was  represented  (see  figure  6).  In  this  way  we  could  compare  a  pilot’s  perception  of  the 
touchdown  point  to  what  we  measured  in  the  simulation. 


Figure  6.  Touchdown  Card 

Finally,  at  the  end  of  all  of  the  trials  the  pilots  were  asked  to  rank  order  the  three  display  configurations,  with  the 
configuration  allowing  the  best  SA  ranked  as  M  Pilots  were  asked  to  provide  rankings  for  two  phases  of  the  approach 
and  landing:  1)  descent  point  to  transition  to  landing  (600'  to  150'  AGL),  and  2)  transition  to  landing  to  touchdown  (105' 
to  O'). 

2.8.4.  Workload 

Three  measures  of  workload  were  collected.  NASA  Task  Load  Index  (TLX)^  and  the  USAFSAM  Workload  Scale  were 
administered  after  each  trial  (see  figure  7). 
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(mental  +  perceptual  activity) 

o|l|2|3|4|  sj  6|7|8|9|l0 

Low  High 

PHYSICAL  DEMAND 

(activate  levers,  knobs,  aircraft  control) 

o|  l|  2|  sj  4|  sj  sj  rj  sj  sj  10 

Low  High 

TEMPORAL  DEMAND 

(time  pressure,  pace  of  tasks) 

o|  i|  2|  sj  4|  sj  el  rj  sj  sj  io 

Low  High 

EFFORT 

(combined  mental  and  physical  workload) 

o|  l|  2|  3|  4|  sj  ej  1  \  sj  sj  10 

Low  High 

PERFORMANCE 

(success  of  performing  the  task) 

o|  l|  2|  3|  4|  sj  sj  rj  sj  dj  10 

Good  Poor 

FRUSTRATION 

(level  of  stress  and  annoyance) 

o|  l|  2|  3|  4|  sj  sj  rj  sj  sj  10 

Low  High 

USAFSAM  Workload  Scale 

1 

NOTHING  TO  DO;  No  system  demands 

2 

LIGHT  ACTIVITY;  Minimum  system  demands 

3 

MODERATE  ACTIVITY;  Easily  managed;  Considerable  spare  time 

4 

BUSY;  Challenging  but  manageable;  Adequate  time  available 

5 

VERY  BUSY;  Demanding  to  manage;  Barely  enough  time 

6 

EXTREMELY  BUSY;  Very  difficult;  Nonessential  tasks  postponed 

7 

OVERLOADED;  System  unmanageable;  Essential  tasks  undone;  Unsafe 

Figure  7.  NASA  TLX  {top)  and  USAFSAM  WorkJoad  Scale  (bottom)  cards 

Finally,  as  with  SA,  at  the  end  of  all  trials,  pilots  were  asked  to  rank  order  the  three  configurations  with  respect  to 
workload.  In  this  case, '  1 '  was  to  indicate  the  highest  workload. 

3.  RESULTS  AND  ANALYSES 


3.1.  Landing  performance 

The  first  objective  of  successfully  touching  down  in  the  500'  assault  landing  zone  with  a  90%  success  rate  was  not  met. 
As  table  1  below  indicates,  the  success  rate  for  assault  zone  landings  ranged  from  16.7%  to  79.2%.  No  condition  resulted 
in  a  90%  success  rate.  Further  analysis  of  the  touchdown  data  using  a  chi-square  procedure  show  that  the  variables  are 
not  associated  with  each  other. 


VMC 

IMC 

Runway 

width 

75' 

150' 

300' 

AvgvMC 

75' 

150' 

300' 

AvgiMC 

Avg 

HUD 

MIS 

79.2% 

62.5% 

20.8% 

54.2% 

58.3% 

66.7% 

54.2% 

59.7% 

57.0% 

CAS 

79.2% 

37.5% 

16.7% 

44.5% 

62.5% 

62.5% 

50.0% 

58.3% 

51.4% 

FORRS 

70.8% 

62.5% 

58.3% 

63.9% 

45.8% 

50.0% 

70.8% 

55.5% 

59.7% 

Table  1.  Percentage  of  landings  in  the  500'  box 


However,  looking  at  the  error  scores  for  lateral  touchdown,  the  main  effects  for  runway  width  and  HUD  configuration, 
and  the  interaction  of  HUD  and  weather  were  statistically  significant.  The  FORKS  was  significantly  better  than  the  MIS 
or  the  CAS  in  terms  of  lower  errors.  The  75'  runway  had  significantly  lower  mean  errors  than  either  of  the  other  two 
runway  widths,  and  the  mean  error  on  the  150'  wide  runway  was  significantly  lower  than  mean  error  on  the  300'  wide 
runway.  See  figures  8  and  9.  _ 
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Figure  8.  Main  effect  for  runway  width  lateral  — rmse  (ft)  Figure  9.  Main  effect  for  HUD  configuration  —  rmse  (ft) 

In  the  HUD  by  weather  interaction,  lateral  error  was  different  between  FORKS  and  MIS  depending  on  the  weather 
condition.  For  MIS,  lateral  error  was  slightly  better  in  VMC  than  IMC,  but  for  FORKS,  the  error  was  lower  for  IMC  than 
VMC.  However,  FORKS  was  still  better  than  MIS  overall  (see  figure  10). 
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Figure  10.  HUD  configuration  by  weather  interaction  —  rmse  (ft) 
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In  the  longitudinal  direction,  there  was  a  significant  interaction  between  HUD  configuration  and  weather  (see  figure  1 1). 
Looking  at  the  contrasts,  the  difference  occurred  between  the  FORKS  and  the  CAS.  CAS  in  IMC  had  slightly  better  error 
than  it  did  in  VMC,  while  FORKS  in  VMC  outperformed  itself  in  IMC.  While  the  interaction  was  significant,  the 
FORKS  absolute  means  were  better  under  both  weather  conditions  than  CAS. 
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Figure  1 1:  HUD  configuration  by  weather —  rmse  longitudinal  (ft) 

There  were  no  significant  main  effects  or  interactions  for  vertical  velocity, 

3.2.  Situation  Awareness 

The  China  Lake  SA  scale  results  revealed  a  significant  interaction  for  runway  width  and  weather  only.  There  were  no 
significant  display  configuration  effects.  The  overall  mean  was  2.28  with  a  standard  deviation  of  .832.  Across  all  18 
conditions,  the  conditional  means  ranged  from  2.00  to  2.75,  indicating  that  SA  was  rated  on  average  between  "Adequate’ 
and  "Good’  in  all  conditions. 

In  terms  of  touchdown  location  situation  awareness,  the  percent  correct  scores  for  the  various  conditions  are  shown 
below  in  table  2.  If  a  pilot  touched  down  in  the  box  and  indicated  so,  their  response  was  scored  as  correct.  Likewise,  if 
they  touched  down  outside  the  box  and  indicated  so,  their  response  was  scored  as  correct.  Only  when  their  responses  did 
not  match  the  touchdown  location  were  the  responses  scored  as  incorrect. 


VMC 

IMC 

Runway 

width 

75' 

150' 

300' 

AVgvMC 

75' 

1  150' 

300' 

AvgtMC 

Avg 

HUD 

MIS 

83% 

67% 

42% 

64% 

79% 

71% 

71% 

74% 

69% 

CAS 

75% 

46% 

37% 

53% 

63% 

63% 

50.0% 

59% 

56% 

FORRS 

71% 

79% 

71% 

74% 

58% 

67% 

96% 

74% 

74% 

Table  2:  Percent  correct  for  touchdown  SA 

For  the  configuration  rankings  for  SA  provided  by  the  pilots,  the  Friedman  test  for  rank  order  data  was  used  for  the  two 
phases  of  flight.  For  each  phase,  the  ranks  from  highest  (FORRS)  to  lowest  (MIS)  SA  were  significant.  For  the  descent 
to  transition  phase  the  average  ranks  were  1 .50,  2.00,  and  2.50  for  FORRS,  CAS,  and  MIS,  respectively  {y^  =  6.0,  df  =  2, 
p  =  .05).  For  the  transition  to  touchdown  phase,  the  average  ranks  were  1.17,  1 .83,  and  3.00  in  the  same  order  (j^  =  20.7, 
df=2,p<.01). 

3.3.  Workload 

While  the  main  effects  for  weather  and  runvyay  width  were  significant,  and  HUD  configuration  was  not,  the  interesting 
result  was  that  the  three-way  interaction  was  significant  (Fg,  86  =  2.198.,  p  =  .035).  The  contrasts  showed  that  the 
significant  differences  in  TLX  and  USAFSAM  scores  occurred  between  the  FORRS  and  the  MIS.  The  overall  trend  was 


for  perceived  workload  to  decline  for  all  three  configurations  with  increasing  runway  width  and  greater  visibility. 
However,  in  IMC  for  the  150'  and  300'  runway  widths,  the  FORRS  provided  workload  scores  equal  to  or  better  than  the 
MIS  and  CAS  in  VMC.  Only  in  the  75'  runway  condition  were  all  three  HUD  configurations  equal  regardless  of 
visibility. 

The  rank  ordering  of  the  three  configurations  according  to  perceived  workload  yielded  no  significant  rater  agreement  for 
either  the  descent  to  transition  or  transition  to  touchdown  phases. 

4.  DISCUSSION 


4.1.  Test  Objective  1 

As  the  results  show,  the  goal  of  90%  in  the  box  performance  was  not  reached  with  any  of  the  display  configurations. 
While  there  was  no  difference  for  in  the  box  performance  among  the  displays,  there  were  some  differences  in  the  lateral 
and  longitudinal  errors  of  each  configuration  within  the  box. 

4.2.  Test  Objective  2 

There  were  no  instances  of  exceeding  the  600'/min  sink  rate  at  touchdown.  Thus,  there  were  no  hard  landings  that  would 
have  resulted  in  a  maintenance  event.  The  concerns  raised  in  the  1998  study  were  not  replicated  in  this  study  under  the 
baseline  configuration  (MIS).  Neither  the  commercial  (CAS),  nor  the  experimental  (FORRS)  configurations  provided 
any  improvement  in  the  maximum  sink  rate  at  touchdown.  This  may  indicate  that  the  concerns  about  hard  landings  using 
a  millimeter  wave  image  were  unfounded. 


4.3.  Test  Objective  3 

The  results  for  SA  show  that  the  FORRS  configuration  provided  the  greatest  amount  of  situation  awareness  about  where 
the  aircraft  touched  down.  However,  if  one  puts  it  in  grade  school  terms,  FORRS  only  earned  a  grade  of  ‘C’. 
Additionally,  MIS  and  CAS  scored  even  lower  overall. 

4.4.  Test  Objective  4 

The  results  for  TLX  and  USAFSAM  indicate  that  while  the  greater  amount  of  information  provided  by  both  the  CAS  and 
FORRS  reduced  perceived  workload,  the  reduction  was  mediated  somewhat  by  the  runway  width.  No  additional  size 
cues  were  provided  by  the  more  complex  display,  the  runway  edges  were  Just  made  more  salient. 

Clearly  we  have  more  work  ahead  to  improve  the  ‘in  the  box’  performance  for  aircraft  attempting  an  assault  landing 
using  only  sensor  imagery  and  symbology.  While  the  display  configurations  performed  only  slightly  better  than  flipping 
a  coin  for  landing  in  the  box,  both  the  minimum  impact  and  the  experimental  configurations  were  much  better  at  letting 
the  pilots  know  they  were  missing  the  box  almost  half  the  time.  One  possibility  is  to  provide  a  standard  size 
object/symbol  in  the  scene  that  affords  pilots  the  opportunity  to  visually  compare  the  true  (sensed)  runway  size  with  a 
known  standard.  Thus,  a  not  only  could  the  runway  outlines  help  with  locating  and  lining  up,  but  the  standard  sized 
object  could  give  an  indication  that  the  runway  is  narrower  or  wider  than  is  typical. 

As  far  as  the  flare  cue  symbology,  results  indicate  that  the  sink  rate  may  not  be  so  problematic,  but  that  completing  the 
touchdown  maneuver  is  still  an  issue.  Several  pilots  commented  that  they,  despite  being  instructed  to  use  the  flare  cue 
symbology,  tended  to  ignore  the  symbology  and  “go  visual”  using  the  sensor  image.  Indeed,  the  flare  cue  symbology  is 
probably  most  often  used  during  instrument  maneuvers  when  there  is  little  scene  to  by  which  to  “go  visual.”  This  may 
indicate  that  a  flare  cue,  when  used  with  an  image,  needs  to  be  integrated  with  the  scene,  rather  than  forcing  the  pilot  to 
split  attention  between  the  symbology  and  the  image. 

As  of  this  writing,  a  follow-up  study  is  underway  to  address  the  issues  raised  in  this  report. 
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